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Over  the  last  three  years,  we  have  carried  out  a  number  of  projects  under  this 
grant,  relating  to  the  gating  and  processing  of  information  in  the  mammalian 
visual  pathway.  Specifically,  they  have  involved  1.)  a  detailed  computational 
investigation  of  the  early  visual  system  of  the  cat  using  both  analytical  as 
well  as  numerical  tools,  2.)  a  study  of  the  dynamic  properties  of  2-D  networks 
of  coupled  oscillatory  neurons  and  3.)  the  outline  of  a  first  neurobiological 
theory  of  visual  awareness. 
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Over  the  last  three  years,  we  have  carried  out  a  number  of  projects  under  this  grant, 
relating  to  the  gating  and  processing  of  information  in  the  mammalian  visual  pathway. 
Speciflcally,  they  have  involved  1.)  a  detailed  computational  investigation  of  the  early 
visual  system  of  the  cat  using  both  analytical  as  well  as  numerical  tools,  2.)  a  study  of  the 
dynamic  properties  of  2-D  networks  of  coupled  oscillatory  neurons  and  3.)  the  outline  of 
a  first  neurobiological  theory  of  visual  awareness.  The  research  was  carried  out  by  myself 
in  collaboration  with  three  post-doctoral  fellows  in  my  laboratory,  Drs.  Dan  Kammen, 
Ernst  Niebur  and  Florentin  Worgdtter,  as  well  as  with  three  outside  collaborators.  Prof. 
John  Kulli  from  the  Department  of  Anesthesiology  at  the  Medical  School  of  the 
University  of  Rochester,  Dr.  Francis  Crick  at  the  Sailk  Institute  in  La  Jolla  and  Prof. 
Heinz  Schuster  from  the  Department  of  Hiysics  at  the  University  of  Kiel  in  Germany. 

Modeling  the  early  visual  system  of  the  cat 

In  our  long-term  effort  to  understand  the  early  visual  system  of  mammals,  we  constructed 
a  simulator  of  a  5°  by  patch  of  the  visual  field  in  the  adult  cat,  by  simulating  the 
underlying  neurons  in  the  retina,  lateral  geniculate  nucleus  and  layer  IV  of  the  primary 
visual  conex  (Wehmeier  et  al.,  1989;  Worgdtter  and  Koch,  1991).  The  current  model 
includes  over  15,000  neurons  and  4,000,000  synapses.  The  simulator  was  implemented 
on  workstations  and,  when  the  execution  time  became  prohibitively  long,  on  the 
Connection  Machine  CM-2.  The  parallel  structure  of  the  early  visual  system  proved  to  be 
ideally  suited  for  the  implementation  on  this  massively  parcel  machine.  Our  approach 
enabled  us  to  test  several  ideas  about  how  feature  extraction  in  the  first  stages  of  the 
visual  system  is  accomplished.  The  three  main  results  are  as  follows:  1.)  A  feed  forward 
excitatory  bias  from  LGN  to  VI  as  postulated  by  Hubei  and  Wiesel  (1962)  is  neither 
sufficient  nor  robust  enough  to  explain  the  observed  orientation  tuning  of  area  17  simple 
cells.  2.)  We  investigate  a  number  of  different  intra-cortical  inhibitory  schemes 
superimposed  onto  the  Hubei  and  Wiesel  excitatory  bias.  The  most  robust  scheme  and  the 
one  in  agreement  with  both  anatomy  and  physiology  is  a  local  inhibition  (iso-mientation 

inhibition)  in  addition  to  a  homogeneous  inhibition  extending  over  500  |im  (circular 
inhibition;  every  cortical  cell  receives  inhibition  from  cells  on  an  annulus  with  this 
radius).  3.)  Spontaneous  direction  selectivity  emerges  from  our  mixed  excitatory- 
inhibitc^  wiring  scheme,  although  we  did  not  specific^ly  include  asymmetric  inhibitory 
connections.  We  showed  that  this  is  caused  by  inherent  asymmetries  due  to  the  mapping 
of  periodic  orientation  columns  onto  the  2-D  cortical  surface  (Wdrgdtter  et  aL  1992). 

We  further  investigated  this  last  point  using  analytical  methods.  The  effects  we  observed 
in  our  large-scale  simulations  result  from  the  interaction  between  intracorticaJ 
interactions  and  the  underlying  column  structure  which  is  generated  by  geniculate 
afferents.  In  order  to  gain  a  deeper  understanding  of  these  phenomena,  we  stripped  the 
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cortical  cells  of  all  their  spatio-temporal  properties  with  the  exception  of  their  (mentation 
selectivity  and  arranged  them  according  to  simple  column  structures.  For  straight 
orientation  columns  we  could  then  compute  analytically  the  sharpening  of  the  orientation 
selectivity  which  results  from  the  proposed  intracorticd  interaction  scheme.  This  proved 
the  effectiveness  of  the  proposed  circular  inhibition  scheme.  Although  straight  orientation 
colunms  are  an  oversimplification,  the  results  hold  for  nK>re  realistic  artificial  systems 
and  also  for  experimentally  observed  cortical  colunm  structures  (Niebur  and  Wdrgdtter, 
1990a,b). 


Temporal  Dynamics  of  Interacting  Neuronal  Populations 

We  have  been  greatly  intrigued  by  the  experimental  findings  of  30-70  Hz,  stimulus- 
induced,  semi-synchronous  neuron^  oscillations  in  the  visual  cortex  of  cats.  Particularly 
noteworthy  is  the  fact  that  neurons  up  to  10  mm  away  can  oscillate  synchronously,  with 
no  detectable  phase-shift,  if  stimulated  by  a  single  object  in  coherent  motion.  Given  the 
great  relevance  we  attach  to  synchronously  firing  neuronal  populations  (see  CMck  and 
K(x;h,  1990  and  below),  we  investigated  the  ability  of  various  networks  of  oscillating 
neurons  to  rapidly  phase-l(x:k  to  each  other.  We  first  explored  the  phase  coupling  in  two 
“extreme”  ty^s  of  networks:  nearest-neighbor  coupling  and  global  feedback  where  every 
cell  communicates  with  every  other  cell  through  a  central  “comparator”  ncxle  (Kanimen, 
Holmes  and  K<x:h,  1990). 

This  model  led  directly  to  studies  of  more  realistic  two-dimensional  networks:  we  used  a 
CM-2  Connection  Machine  to  simulate  a  128  by  128  grid  of  16,384  cells  under  a  variety 
of  stimulation  patterns  (Niebur,  Kammen  &  Koch,  1991).  To  explore  phase  l(x;king 
between  extended  or  spatially  separated  groups  of  cells,  we  stimulate  the  network  with  a 
series  of  “bars”  that  receive  similar  input.  To  simulate  biologically  observed  connection 
schemes,  we  fu^t  connected  each  cell  to  all  of  its  neighbors  within  a  given  distance  by  a 
synapse  whose  strength  is  a  Gaussian  function  of  the  distance  between  the  cells.  We  then 
mcxlifled  the  Gaussian  scheme  by  deleting  nearly  99%  of  the  connections,  but  setting  the 
remaining  1%  to  equal  weight,  and  discovered  that  leaking  and  phase  coherence  was 
dramatically  improved  (Niebur  et  al  1991b).  This  always  involved  large  parts  of  the 
system  and  therefore  it  is  a  plausible  mechanism  for  long-range,  complete 
synchronization.  We  showed  analytically  and  numerically  that  the  development  of  long- 
range  synchronicity  is  a  general  property  of  dynamical  systems  consisting  of  sparsely 
connected  oscillators  (Niebur  et  al,  1991a).  On  the  other  hand,  much  of  the  local 
information  is  destroyed  by  this  process  and  therefore  tasks  like  figure-ground 
segregation  (which  have  been  put  forward  as  a  possible  function  of  these  oscillati(Xis) 
seem  to  require  relatively  localized  connections,  as  the  Gaussian  connectivity  scheme 
provides.  Our  results  suggest  that  long-range  synchronization  of  neurd  activity,  as  is 
observed  in  the  neurophysiological  recordings,  is  likely  to  be  subserved  by  pathways  of 
quite  spare  yet  long-range  connections.  Recent  studies  of  cortical  micrcxnrcuitry  indicate 
that  while  the  majority  of  connections  are  locally  dense,  or  Gaussian,  there  does  exist  a 
small  population  of  pyramidal  cells  with  long  axons  that  make  contacts  on  cells  up  to  3 
mm  away.  Our  simulation  results  therefore  tind  a  role  for  both  kinds  of  connections. 

Transmission  of  information  in  the  conex  is  subjea  to  considerable  time  delays  (from  a 
minimum  of  about  I  ms  to  nuuiy  ms)  between  the  time  an  actiem  potentiid  is  generated  in 
the  sending  cell  and  the  time  it  has  an  effect  on  the  receiving  cell.  Although  this  delay, 
which  is  due  to  the  finite  propagation  vel(x;ity  along  nerve  processes  and  to  the  finite 
transmission  time  across  chemical  synapses,  is  usually  significantly  smaller  than  the 
peri(xl  of  the  40  Hz  oscillations,  it  is  certainly  not  negligible  and  we  set  out  to  explore  the 
consequences  of  small  delays  on  coupled  oscillations  in  cortical  systems.  We  u^  again 


systems  of  16,384  oscillators  simulated  on  the  Connection  Machine  CM-2,  this  time 
interacting  by  nearest  neighbor  connections.  We  found  two  surprising  results:  1)  even  if 
the  delays  are  snoall  compared  to  the  average  period  of  the  oscillators,  the  average 
frequency  of  the  system  is  lowoed  considerably  in  the  case  of  strong  interactions  (which 
is  the  case  we  are  interested  in),  and  2)  for  larger  delays,  the  system  has  numerous 
metastable  states  with  higher  frequencies  besides  tiie  one  stable  state  with  low  frequency. 
We  then  developed  an  analytical  nKxiel  of  the  system  and  verified  that  both  effects  can  be 
understood  in  this  model  (Niebur,  Schuster  &  Kammen  1991, 1992). 

While  the  30-70  Hz  oscillations  have  been  unambiguously  established  in  cat  cortex,  their 
existence  is  more  controversial  in  the  cortex  of  the  awake  and  behaving  macaque 
monkey.  Thus,  it  is  all  the  more  rewarding  that  we  have  been  recently  able  to  show  that  in 
a  cortical  network  based  on  the  canonical  microcircuit  of  Douglas  and  Martin,  strong 
“burst”  synchronization  can  occur  without  oscillating  neurons  (Koch  and  Schuster,  1992; 
Schuster  and  Koch,  1992).  The  network  consists  of  all-to-all  excitatory  coupled  binary 
neurons  with  global  inhibition.  We  prove  rigorously  that  for  random  input  signals,  the 
output  of  the  network  consists  of  synchronized  bursts  with  apparently  random 
intermissions  of  noisy  activity.  Our  results  imply  that  synchronous  bursts  can  be 
generated  by  a  simple  neuronal  architecture  which  amplifies  incoming  coincident  signals. 
This  synchronization  process  is  accompanied  by  dampened  oscillations  which,  by 
themselves  do  not  play  any  constructive  role  in  this  and  can  therefore  be  considered  to  be 
an  epiphenomenon. 


The  neuronal  basis  of  visual  awareness 

We  published  several  papers  (Crick  and  Koch,  1990,  1991;  Koch  and  Crick,  1991)  that 
outline  a  testable,  neurobiological  theory  of  consciousness.  In  particular,  we  suggest  that 
visual  awareness  is  a  favorable  form  of  consciousness  to  study  neurobiologicdly.  We 
propose  that  it  takes  two  forms:  a  very  fast  form,  linked  to  iconic  memory,  that  may  be 
difficult  to  study;  plus  a  somewhat  slower  one  involving  visual  attention  and  short-term 
memory.  In  the  slower  form  an  attentional  mechanism  transiently  binds  together  all  those 
neurons  whose  activity  relates  to  the  relevant  features  of  a  single  visual  object.  The 
neuronal  correlate  of  binding  is  temporal  synchrony  of  firing;  i.e.  aU  neurons  that  code  fOT 
different  attributes  of  the  same  object  fire  synchronously.  This  synchronous  firing  may 
involve  oscillatoiy  neuronal  activity  in  the  40-70  Hz  range.  The  synchronous  firing  then 
activates  a  transient  short-term  (working)  memoiy  (for  a  short  and  succinct  summary  of 
this  sec  Koch  and  CMck,  1991). 


Future  Research 

We  are  pursuing  these  ideas  at  several  levels.  In  a  joint  collaboration  with  Prof.  Bill 
Newsome  at  Stanford  University  Medical  School,  we  are  analyzing  the  temporal 
dynamics  of  single  and  multi-unit  recordings  from  extrastriate  visual  cortex  in  the  awake 
and  behaving  macaque  monkey  using  various  statistical  techniques.  The  aim  is  to  relate 
the  fine  structure  of  temporal  fuing  (e.g.  40  Hz  oscillator  activity,  bursting,  synchronized 
activity)  to  the  known  and  measured  behavior  of  the  monkey  (Bair,  Koch,  Newsome  and 
Britten,  1992). 

Our  models  of  geniculate  and  visual  cortex  are  becoming  more  refined.  In  collaboration 
with  Drs.  Kevan  Martin  and  Rodney  Douglas  from  (Oxford  University  (R.D.  spends  3 
months  each  year  in  our  laboratory  at  Caltech),  we  are  elaborating  on  our  earlier  detailed 
models  of  cat  conex,  by  using  more  realistic  single  cells  models  and  the  mme  detailed 
knowledge  of  cortical  mircocircuit  that  is  now  available.  Again,  one  aim  here  is  to  study 


and  understand  in  more  detail  the  onset  and  development  of  temporal  correlation  among 
neurons  (Suarez,  Douglas  and  Koch,  1992). 

To  what  extent  is  the  temporal  structure  of  neuronal  firing  involved  in  attendonal  gating 
in  higher  ctxrtical  areas?  We  are  pursuing  this  issue  by  constructing  a  detailed  model  of 
the  cortical  and  subcortical  structures  underlying  selective  visual  attention  in  primates 
(pulvinar,  colliculus,  VI,  V2,  V4,  infero-temporal  cortex,  parietal  cortex,  etc.)  and 
replicating  the  classical  experiments  of  Moran  and  Desimone  (1985).  The  performance  of 
such  a  firmly  neuronal  based  model  must  also,  however,  be  in  agreement  with  the 
extensive  psychophysical  literature  on  the  effects  of  attention  (e.g.  “serial”  versus 
“parallel”  search;  effortless  segregation  of  certain  textures  and  patterns;  see  the  work  by 
Julesz,  Treisman,  and  Posner). 

Thus,  the  three  strands  of  the  research  funded  by  AFOSR  under  this  grant  are  coming 
together  in  our  ongoing  and  future  efforts  to  understand  at  the  neurobiological  level  how 
attentional  gating  of  visual  information  occurs  in  the  visual  pathways  of  monkeys  and 
humans  and,  ultimately,  how  perception  and  awareness  is  expressed  at  the  level  of 
neuronal  firing.  This,  of  course,  is  the  principal  topic  of  our  current  research  proposal. 
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